Robust Audio-Visual Person Verification Using Web-Camera Video

This thesis examines the challenge of robust audio-visual person verification using data recorded in multiple environments with various lighting conditions, irregular visual backgrounds, and diverse background noise. Audio-visual person verification could prove to be very useful in both physical and logical access control security applications, but only if it can perform well in a variety of environments. This thesis first examines the factors that affect video-only person verification performance, including recording environment, amount of training data, and type of facial feature used. We then combine scores from audio and video verification systems to create a multi-modal verification system and compare its accuracy with that of either single-mode system. Thesis Supervisor: Timothy J. Hazen Title: Research Scientist, Computer Science and Artificial Intelligence Laboratory Thesis Supervisor: James R. Glass Title: Principal Research Scientist, Computer Science and Artificial Intelligence Laboratory